This invention relates to a method and a device for managing image information in a monitoring system, in which the management of image information comprises both processing of image information and transmission of processed image information. The invention also relates to a monitoring module and a computer program product.
Monitoring of various public places, installations and premises is becoming increasingly important as they contain increasingly valuable equipment such as, for example, computers which in turn contain information that is very important to the owner, and perhaps also confidential. There is also a need for monitoring, for example, of baggage at airports and also in private dwellings. It is normally desirable for the monitoring information to be displayed immediately and accurately in order to enable the appropriate measures to be taken quickly.
In order to meet these needs, there are different types of monitoring systems. One type of monitoring system according to prior art technique normally comprises a monitoring center and a number of monitoring modules. Each monitoring module is connected to the monitoring center via communication cables. A monitoring module can comprise a video camera and an infrared detector which are connected to each other via a cable. It operates by the infrared detector detecting a movement and the video camera recording an image. The recorded image is then sent to the monitoring center. A problem with infrared detectors is that they are relatively easy to mislead, which makes the monitoring unreliable. For example, a sheet of glass can be placed in front of the detector, which means that heat changes are not detected and the video camera will not start recording. A further problem with this type of monitoring system is that in many cases the recorded images do not provide sufficient information about what caused the alarm. This can occur when, for example, alarm situations which have been caused by high temperatures or sabotage are not caught by the camera. As a result, there remains uncertainty as to whether it is a genuine alarm.
In order to solve this problem, it has been proposed that images should be recorded continually. The recorded image can be compared with a reference image, which is normally a background image of the area which is being monitored. If there is any difference between these images and if a change has occurred in a recorded image, these images are transmitted to a manned monitoring center. A system operator at the monitoring center looks at the transmitted image and from that makes a decision regarding the monitoring situation. A number of situations which the infrared detector would not detect will be detected by this method. A disadvantage of this technique is that a lot of information must be transmitted, as there are many situations, such as changes in lighting conditions, which can cause changes in the recorded image. It takes a long time to transmit the image. Before the transmission is carried out, the image or images are compressed using, for example, JPEG or MPEG. A transmission, which is a video transmission of consecutive images of the monitoring site, is limited to a certain number of images per second. If a mobile telephone of the GSM type with a transmission speed of 9.6 kbit/s is used as the receiver for the monitoring information, it can take 20-100 seconds before the transmitted image becomes clear and decipherable. By means of compression of the images, it is possible to reduce this to 3-4 seconds per image. If there is further compression, the image becomes so unclear that events become difficult or even impossible to make out. This means that if the degree of compression is increased in order to reduce the bandwidth, the image quality finally becomes so poor that it is not possible to draw the correct conclusions from the image. As an alternative, the image frequency can be reduced and fewer images sent. However, this does not provide satisfactory monitoring, as much can happen between two consecutive images and here too it can be difficult to draw conclusions about actual movements. Problems also arise when it is wished to store monitoring information in the monitoring module. In order to be able to store all the information which is needed to determine whether there is an alarm situation, a large storage capacity is required, which results in the equipment being expensive and its complexity increasing.
Patent application WO 98/28706 describes a security system which comprises a number of cameras which are arranged to record images and to transmit these images to a monitoring station in which digital image processing is carried out. The monitoring station carries out processing to determine whether there is an alarm situation or not, and if there is an alarm situation a signal is forwarded indicating whether it is a human-related alarm or an unknown alarm.
U.S. Pat. No. 5,666,157 describes a monitoring system which detects unusual events, which can be movements which are interpreted as personal attacks. Recorded images are analyzed, objects are detected and the movement characteristics of the object are calculated. Based on the movement characteristics, a decision is reached which indicates to what extent a movement is judged to have a criminal intention.
An object of the invention is therefore to make possible reliable and cost-effective monitoring.
This and other objects which will be apparent from the following description are achieved by a method for managing image information in a monitoring system, which monitoring system comprises at least one monitoring module for monitoring a monitored location and a remote recipient unit, comprising the steps of recording an image of the monitored location by means of the monitoring module, segmenting a distinct region in the monitoring module by comparing the recorded image with a reference image, creating an outline shape in the monitoring module which represents the edge of the distinct region, transmitting data which represents the outline shape to the recipient unit, recreating the outline shape in the recipient unit by means of the said transmitted data, and displaying the outline shape visually in the recipient unit.
The monitoring module can continually record images of the monitored location. The monitored location is limited among other things by the components of the monitoring module for recording images. If a change has occurred in a recorded image in comparison to a reference image, such as a person or an animal having entered the image, this object is segmented by means of a number of different algorithms. The reference image is created with one or more algorithms from one or more previous images, one or more background images of the monitored location or a combination of both. The advantage of the comparison is that moving objects can be processed further and stationary objects, such as tables and chairs, which are in the monitored location can be excluded. This means that a distinct region contains interesting information about events in the monitored location. An object which is segmented from an image is represented by this distinct region.
An outline shape is created. An outline shape is a stylized representation of the edge of the distinct region. Data representing this outline shape occupies little bandwidth and is therefore suitable for transmission. The recipient unit is the unit which receives data representing the outline shape and processes it in an appropriate way. The monitoring module and the recipient unit interpret the transmitted information in the same way. They have the same mathematical model of the outline shape. That is to say that the monitoring module processes the outline shape in accordance with a mathematical model and produces data representing the outline shape and transmits this data to the recipient unit. The recipient unit which receives the data can, if so required, have knowledge of which mathematical model is used and can recreate the outline shape for visual display. The recipient unit can, for example, be located in a manned monitoring station so that the outline shape is displayed on a screen to an operator. The operator makes a decision regarding whether it is an alarm situation and can then take appropriate measures.
A great advantage of transmitting the outline shape of an object is that, if the object is a person, its privacy is preserved. This can be very important, as special permission is often required to erect monitoring cameras, precisely in order to protect personal privacy. In addition, it should be possible to use the technique in private homes and it can then be desirable for the people who live in homes with monitoring devices not to be recorded on video or photographed, as, for example, these pictures could be misused.
In one embodiment, the step of creating an outline shape comprises the steps of creating an outline image of the distinct region and of fitting the outline shape to the outline image.
The outline image can consist of a sequence of points along the edge of the distinct region. An advantage of using the outline image is that it is easy to produce the outline shape in this way.
In another embodiment, the step of creating the outline image comprises the step of following the edge of the distinct region with a search function, such as a clock-hand algorithm.
The outline image is extracted in this way from around the region. Following this, a shape is fitted mathematically to the sequence of points in the outline image.
An embodiment comprises the steps of classifying the distinct region in the monitoring module from at least one characteristic of the distinct region, such as size and/or shape, the classification controlling the transmission of said data.
Dependent upon the classification of the distinct region, it is determined whether it is of interest for forwarding to the recipient unit. It can, for example, be the case that an area is monitored and that the area is also guarded by a dog. Data concerning the outline shape of the dog is thus not to be forwarded.
An embodiment further comprises the step of comparing in the monitoring module particular characteristics belonging to the distinct region, such as one or more characteristics of the type: size and shape, with corresponding characteristics belonging to a region segmented from at least one previously recorded image, the associated movement history of the distinct region being recorded if the characteristics conform to the extent that they are determined to represent the same object. The recording thus takes place by matching with the distinct regions of previously recorded objects. Two distinct regions recorded at different times are said to represent the same object if they meet certain predetermined matching criteria. For example, characteristics of the distinct region can be compared, such as its physical size in the image, and if they correspond to a particular extent it is determined that it matches. In this way, a movement history of a distinct region can be produced. For example, the speed and direction of movement can be worked out.
Another embodiment further comprises the steps of classifying the distinct region in the monitoring module based on the recorded movement history of the distinct region, the classification controlling the transmission of said data. The movement information is classified in order to determine whether the distinct region is an alarm object or not. If the distinct region is classified as an alarm object, the outline shape is to be transmitted to the recipient unit. The classification can, for example, be carried out based on how quickly an object moves or the direction of movement of the object. For example, if an object moves short distances back and forth, it can be a tree or a curtain blowing in the wind. These movements are thus not to be classified as alarm objects. In this way the number of false alarms is reduced. In addition, the amount of transmitted information is further reduced.
An embodiment further comprises the steps of classifying the distinct region in the monitoring module based on at least one characteristic belonging to the distinct region, such as size, shape and/or recorded movement history, the classification controlling the transmission of said data.
In one embodiment, data representing the movement history is transmitted to the recipient unit together with said data representing outline shape and is recreated for visual display.
The visual display can, for example, be carried out by consecutive outline shapes being combined into a moving sequence. This sequence can be combined either in the monitoring module or in the recipient unit. By having the ability to display the movement history of the outline shape which reflects the behavior of the transmitted object, an operator can, for example, more easily make an evaluation concerning the alarm situation. The transmission of the movement history only needs to involve a very small increase in the amount of data transmitted. While privacy protection is retained, it is possible to have reliable monitoring, as it is a person""s shape, as shown by the outline, and pattern of movement which are important when someone is to draw a conclusion based on a visual display as to whether there is an alarm situation or not. The movement history can, for example, also be represented by a movement vector showing the direction of movement of the object and its speed.
In one embodiment, said data is transmitted only if the distinct region is classified as human-related.
This makes possible, for example, burglary monitoring. The data which is transmitted can be displayed visually at a recipient unit and an evaluation of the human-related outline shape can be carried out. For example, it can be decided whether the outline shape is an actual alarm object, based on where on the monitored location the outline shape is located. For example, at one side of the monitored location there can be a road where people can pass by without causing an alarm situation. If the movement information is also transmitted, the behavior of the object can form the basis for a visual evaluation of whether there is an alarm situation. The method according to the invention is particularly suited to monitoring people, as it provides privacy.
Another embodiment further comprises the step of storing the data which is to be transmitted to the recipient unit in the monitoring module prior to transmission.
The outline shape requires little storage space and can therefore be stored in a memory in the monitoring module. As this storage does not require much space, the cost of the monitoring module is reduced. The possibility of storage in a memory in the monitoring module is a great advantage if, for example, a fault arises in the recipient unit or in the communication between the monitoring module and the recipient unit or if the recipient unit becomes overloaded, as the outline shape can be sent at a later time when the function is restored. If there is a system operator, he can also be allowed to retrieve the outline shape from the monitoring module for analysis afterwards, if, for example, transmission is not possible. There can be several monitoring modules which cooperate to store the information. For example, if the movement history is to be transmitted, this can also be stored before transmission.
One more embodiment further comprises the step of transmitting supplementary monitoring information, such as audio recording, partial lines and intensity area within the distinct region, to the recipient unit.
An advantage of this is, for example, that the operator can request more information if he finds it difficult to make a decision based on the information which has already been displayed. The supplementary monitoring information can consist, for example, of one or a few images of the distinct region. This supplementary information can be transmitted as a bit-map image. This means that the area that is of interest is segmented out and the image of this interesting area is transmitted. The supplementary monitoring information can also be a sound recording. Another type of supplementary monitoring information can be intensity areas within the distinct region. When this data representing these intensity areas is transmitted, characteristic features within the distinct region are displayed. For example, a division of a person can be carried out into four intensity areas, hair, face, upper body and trousers/lower body. A further type of supplementary monitoring information can be so-called partial lines within the distinct region. The partial line content gives the distinct region more structure and essential information about the texture of the object. Examples of partial lines in a person can be that a chin portion is added so that the head is regarded as part of the rest of the body. It is easier to make out what the outline shape represents.
The system operator can have the ability to increase the amount of supplementary monitoring information at the expense of the bandwidth. An embodiment further comprises the step of displaying the recreated outline shape on a background image in the recipient unit. The background image can be recorded initially when the monitoring module is started up and transmitted to the recipient unit. If required, it can be possible for the alarm operator, for example, to request an update. By sending the background image of the monitored location once to the recipient unit and then only sending the outline shape, the feature is retained that the amount of data sent from the monitoring module to the recipient unit remains small, while at the same time the visual evaluation of the displayed data by an alarm operator is made easier, as the alarm operator has now something as a point of reference for the outline shape and also any movement history.
In one embodiment, the communication between the monitoring module and the recipient unit is carried out by wireless means, for example by mobile telephony.
Wireless transmission makes possible mobility of the recipient unit. This means that if an operator has a recipient unit, the operator does not need to remain stationary. For example, the outline shape can be received by a recipient unit which is arranged in a mobile phone.
In one embodiment, the outline shape is represented by a polygon.
An advantage of using a polygon function is that compression algorithms can be used effectively. A polygon is also good when an alarm operator is to make a visual evaluation of the outline shape.
In one embodiment, the polygon is represented by a number of points, the number of which is variable. By using a smaller number of points, a lower bandwidth is required, but at the same time the quality of the outline shape deteriorates. Other outline shapes can also have similar characteristics. It is a great advantage to be able to select the number of points in the polygon, as the access to bandwidth and required image quality can vary from occasion to occasion and from monitoring module to monitoring module. For example, certain premises which contain very valuable equipment can require extra high verification quality and a larger number of points is then selected to be used for the polygon. There can also be various types of problems which, for example, can cause reduced function in the recipient unit and which at the same time are set against high performance requirements, which means that the bandwidth must be reduced in order that the recipient unit is not to be overloaded. The number of points can then be reduced.
In one embodiment, the outline shape is represented by a spline function.
A spline curve is a curve which is controlled by a number of control points. The position of the control points is adjusted so that the curve coincides as well as possible with the outline in the image. This curve has the advantage of being good for visual evaluation of an alarm situation for the outline shape.
The invention also relates to a device for managing image information, which device comprises at least one monitoring module for monitoring a monitored location and a remote recipient unit, the monitoring module comprising a light-sensitive sensor which is arranged to record an image of the monitored location, a calculating unit which is arranged to segment out a distinct region by comparing the recorded image with a reference image and to create an outline shape which represents the edge of the distinct region, and a communication unit which is arranged to transmit data representing the outline shape to the recipient unit; and the recipient unit being arranged to receive said transmitted data, to recreate the outline shape and to display the outline shape visually.
In one embodiment, the monitoring module and the recipient unit interpret according to the same mathematical model. The advantage of this is that information transmitted from the monitoring module can be interpreted by the recipient unit and possibly also recreated, for example, for visual display.
In another embodiment according to the invention, the device comprises a audio device for transmitting audio information from the monitoring module to the monitoring station.
In an embodiment according to the invention, the communication unit comprises a wireless communication device.
The advantages of the device are apparent from the above discussion of the method. The invention also relates to a monitoring module for monitoring a monitored location comprising a light-sensitive sensor, which is arranged to record an image of the monitored location, a calculating unit which is arranged to segment out a distinct region by comparing the recorded image with a reference image and to create an outline shape which represents the edge of the distinct region, and a communication unit which is arranged to transmit data representing the outline shape to a remote recipient unit.
The invention also relates to a computer program product which comprises program code which is stored on a computer-readable medium and which, when loaded into a computer, carries out one or more of the method steps according to any one of claims 1-16.
Further advantages of the monitoring module and the computer program product are apparent from the above.